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[57] 



ABSTRACT 



An apparatus which remotely visualizes and detects toxic, 
flammable and other gas leaks and enables one to sec gas 
clouds in real-time. A dual band thermal imaging infrared 
video camera fitted with a special filter images gas clouds. 
Image processing is utilized to develop and colorize the gas 
cloud information for display. The gas image is superim- 
posed over a background image provided by a co-located 
visible light video camera. 



39 Claims, 4 Drawing Sheets 
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APPARATUS FOR IMAGING GAS " of these systems is not very large and, perhaps 'the 1 biggest 

disadvantage of all is that the systems cannot image against 

BACKGROUND OF THE INVENTION a sky background. U.S. Pat No. 4,555,627 shows one 

1. Field of the Invention arrangement of a backs catter absorbing gas imaging system. 
This invention relates to infrared radiation detection and, 5 Other systems for detecting gases, chemicals or the like 

more particularly, to infrared video cameras and associated are shown, for example, in U.S. Pat Nos. 3,563,658; 3*662, 

processing systems for visualizing gas clouds on a video 171; 3,715,497; 4,227,210, 4390,785; 4,543,481; 4,670, 

display. 653; 4,725,733; 4,937,477; 4,963,742; 4,963,744; 4,965,447 

2. Background Art and 4,967,276. 

There are literally thousands of chemical processes and & is not uncommon to process the output signals gener- 
systems which need continuous monitoring to ensure the ated b y infrared video cameras to enhance the desired signal 
safety and health of both site workers and the general remove unwanted noise. The prior art processing has 
populace. It is common for chemicals, either during the **** generally limited to integration and subtraction tech- 
manufacturing process or in the bulk distribution thereof, to ^ niques which limit the high noise levels inherent in infrared 
be stored for long periods of time, moved about an area, video cameras. Integration has classically been used to 
transferred from container to container or transported in long reduce the quantity of random noise present in the signal 
pipelines. Unfortunately, it is not uncommon for chemicals However, the chief failing of integration is that it also 
to be stored or transported in deteriorating tanks or pipes. prevents the quicker and more transient features of the 
The chance of spills and accidental releases of chemicals to „ desired signal from being seen. For example, the edges of 
the environment seems to rise every day. There is a clear d°uds and the areas subjected to wind may be effec- 
ted not only for a range of monitoring techniques to deal dve ry integrated from the signal. Subtraction of consecutive 
with early detection and subsequent tracking of accidental frames from each other has also been used to separate the 
spills or leaks, but also for routine monitoring of chemical moving portions in the image from the stationary portions, 
plants and storage areas. « usm 8 me assumption that the gas cloud moves while the 

Current monitoring techniques become inadequate when background image does not A major limitation of subtrac- 

gases are involved because of the ability of most gases to f n ******** ^ °«f n * c gas cloud moves so slowly 

travel unseen far past the point of original detection. T^e ^ * 111 ***** to background, is removed. Subtrac- 

wind alone can make the path of release a matter of guess * on *™ <f° ? othm * to **» the noise when the noise 

work and even the point of origin a matter of doubt What 30 "Semite &™ in its ability to change from frame to frame, 

is needed is a method in which escaping gases can be made The primary difficulty of the prior art techniques is that 

visible to the eye and use the superior ability of human they are not adequate to separate the signal from the noise 

vision to gauge location and general movement Once vis- in a gas imaging system. A vague image of the gas cloud 

ible to the human eye, the gas image can men be subjected might be visible after processing, but it is not sufficient to 

to processes of automation, such as machine vision, to aid in 35 permit accurate and reliable detection, particularly by auto- 

the monitoring and detection process. matic methods. Accordingly, it is an object of the present 

Considerable effort has been expended in the past in the invention to provide enhanced processing techniques on the 

application of thermal imaging to gas detection. Optical 0Ut P ut of m infrared video camera or the like to provide a 

techniques in general, based upon the absorption or emission signal-to-noise ratio and a more accurate image of a 

of infrared radiation by gases and vapor, have been continu- 40 nioving gas cloud. 

ously developed for many years. Today, it is possible to use It is a further object of the present invention to provide a 

laser and non-laser systems to make precise concentration passive, non-laser based, preferably uncooled, infrared 

measurements along beam paths. However, there are few video camera detection system for accurately and inexpen- 

practical systems for obtaining an image of a gas cloud and, sively imaging a moving gas cloud. 

hence, for obtaining a real-time representation of the dynam- 45 „ _™ ^ 

ics of the gas cloud. SUMMARY OF THE INVENTION 

One present technique for measuring gas is known as Accordingly, we have developed an apparatus for gener- 

backscatter absorbing gas imaging, and uses an infrared ating a visible image of a gas cloud against a background 

video camera, together with a laser, to raster scan and scene. The apparatus preferably includes a visible light 

illuminate the scene. The laser beam passes out through the 30 video camera and an infrared video camera, such as a 

camera optics in the reverse direction to the incoming pyioelectric infrared video camera, which view substantially 

radiation and the backscatter radiation from the background the same scene. A chopping filter is associated with the 

(terrain, buildings, etc.) behind the gas cloud is detected by infrared camera and alternately presents first and second 

the camera. The gas is detected by its absorption relative to band pass filters which pass radiation of two different 

the background. These systems offer good sensitivity and 55 infrared wavelengths to the infrared video camera. The first 

signal-to-noise ratios. However, they use a cooled infrared band pass filter passes infrared radiation at a measuring or 

video camera which requires a separate source of cooling, strongly absorbing infrared wavelength for a predetermined 

such as a compressor, a bottle of liquified coolant or a gas. The second band pass filter passes infrared radiation at 

thermoelectric cooler to operate. The cooled camera not only a reference or weakly absorbing infrared wavelength for the 

adds considerably to the expense of the system, but also 60 predetennined~ gas. The apparatus also includes an image 

increases maintenance and reduces the portability of the processor responsive to an output from the infrared video 

system. In addition, there are limitations imposed on the use camera for reducing noise and enhancing an information 

of lasers, an active element in the system, which severely signal generated by a detected gas cloud. An overlay means 

limits the overall utility of these systems. Only gases which combines an output from the visible light camera with an 

have absorption features coincident with the laser (C0 2 , 65 output from the image processor and produces a signal 

He — Ne, etc.) can be detected. The essential laser also adds representing an image of the gas cloud superimposed upon 

to the cost and complexity of the system. Rrescntly. the range the visible image of the background scene. Finally, the 



07/13/2004, EAST Version: 1.4.1 



5,656.813 

3 4 

v > apparatus includes a display, means, such, as a monitor, f or . . . line ; brightness fJ level of the, -infirared^, video camera. 1 1 

displaying the signal generated by the overlay means. response to any drift in said baseline brightness level, the 
It is preferred that the reference wavelength of the second whole picture averager adjusts a black level control in the 
band pass filter be close to the measuring wavelength of the infrared video camera to maintain an overall brightness level 
first band pass filter. The chopping filter can be a rotating 5 therein at a substantially constant level Preferably, the 

disc having separate first and second band pass areas therein, baseline brightness level is generated by simuning the 
such as a thin, circular wafer of a semiconductor substrate amplitude values of all pixels in one or more frames and 
having layers of material deposited thereon to form the band dividing the sum by the number of frames utilized, 
pass filter areas. The band pass filter areas can be formed in 

the shape of nested, outwardly tapering archimedian spirals i° BRIEF DESCRIPTION OF THE DRAWINGS 

separated by about 180°. In a preferred embodiment, the piG. 1 is a schematic diagram showing the spectral 

chopping filter satisfies the equation: wn&iLg principles utilized in the present invention; 

FIG. 2 is a schematic diagram showing the principles of 

**& m >N&kU i5 using a chopping filter in accordance with the present 

where N* is the spectral radiance of the background scene, invention with a pyroelectric vidicon camera; 

X M is the measuring wavelength and A* is the reference FKJ. 3 is a plan view of a preferred chopping filter in 

wavelength. This condition can be achieved by control of the accordance with the present invention; 

density of the filter areas during manufacture. FIG. 4 is a graph showing the Planck radiation function of 

A preferred image processor includes a storage means for 20 the chopping filter shown in FIG. 3; 

storing at least one prior output frame of the infrared video FIG. 5 is a schematic diagram of the preferred image 

camera. The image processor also includes a comparison processor used in the present invention; 

meansfor comr^g me current output fW of me infrared nG 6AandFia ^ „ schem atic diagrams showingthe 

video camera wi* a prior frame in the storage means and g . ^ m ^ W<K | of 

generating a difference signal representing differences 25 5. and 

between the current frame and the prior frame. The image " ' 

processor also includes a convolution means responsive to nG - 7 - 18 a schematic diagram showing a complete gas 
the difference signal from the comparison means for imaging system in accordance with the present invention, 
tetemrining, on a pixel by pixel basis for each frame, nF^rRTrmoM ofthp prpfprpfh 
whether a particular pixel should be further activated based 30 ^^^^^ODnSs^^ 
upon a spatial relationship of activated pixels in a surround- 
ing matrix of pixels, including the particular pixel. Finally, The present invention preferably uses a pyroelectric infra- 
the image processor includes a temporal means responsive ted video camera for detecting die infrared radiation char- 
to an output generated by the convolution means for acti- acterisric of the gas cloud under observation. The pyroelec- 
vating a pixel only if that pixel is activated in the current 35 trie infrared video camera is a passive device, requires no 
frame and was activated in a predetermined number of coolant is relatively inexpensive, requires little maintenance 
consecutive prior frames. and can operate for extended periods of time in many 
The image processor can further include a threshold different environments. A pyroelectric infrared video camera 
means responsive to the difference signal from the compari- operates by changes in optical power absorbed by a pyro- 
son means for supplying to the convolution means only 40 electric crystal which is the sensing element on the faceplate, 
difference signals which exceed a predetermined threshold The pyroelectric infrared video camera can be provided as a 
level The image processor can also include an integrator vidicon version which includes a single, large pyroelectric 
means responsive to the output of the infrared video camera element which is raster scanned or as a recently developed 
for integrating at least two frames of the infrared video array version which includes a 256x256 array of separate 
camera output before supplying the frame for storage in the 45 pyroelectric elements. The array version may prove to be 
storage means and before supplying the current frame to the preferred. Variations with time in the temperature of the 
comparison means. In addition, the image processor can pyroelectric material give rise through me pyroelectric effect 
include a sustainer responsive to an output from the tern- to variations of surface charge. By sampling the charge, 
poral means for maintaining each activated pixel in a frame changes in temperature of the material may be measured. To 
as activated for a predetermined period of time beyond the 50 a first order of approximation, the charge signal is prop or- 
duration of the frame. The image processor can further tional to the rate of change of the temperature of the 
include a control means for establishing: (1) the rate of material. The output signal of a pyroelectric infrared video 
frame integration for the integrator means; (2) die rate of camera in the vidicon version is generated by scanning the 
storage of frames in the storage means; (3) the threshold target material in a television raster by an electronic beam 
level of the threshold means; (4) the spatial filtering method 55 readout and in the array version is generated in a CCD 
used in the convolution means; (5) the number of consecu- readout with each separate element generating a separate 
rive prior frames considered in the temporal means; and (6) pixel signal. 

the period of time of the sustainer. All pyroelectric cameras have a common property which 

In a preferred embodiment the convolution means exam- is different from most other cameras. A steady flux of 

ines a spatial relationship of an nxn array of pixels, such as 60 incident radiation on the detector, i.e., the pyroelectric 

a 3x3 array of pixels, including the particular pixel under material, will cause the detector to reach thermal equilib- 

consideration. In mis preferred embodiment, the particular num. Under these conditions, redistribution of charge 

pixel in a frame is activated only if the three pixels in a through displacement currents will take place until there is 

left-hand column of the pixel array are each activated. The no resultant charge and, hence, zero output signal In order 

image processor can further include, in a feedback loop from 65 to provide a constant output signal, the radiation flux must 

the infrared video camera output to the infrared video be modulated. It is common in some pyroelectric cameras to 

camera, a whole picture averager which monitors the base- provide a rotating chopper which alternately passes the 
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radiation to the camera- and stops the radiation from travel- r ^proccssing^gcnexally referred^ by Science number^, to ' ; 
ing to the camera. Others need to scan the scene to provide generate the output video signal from the pyroelectric cam- 
modulation. In the arrangement disclosed herein, a specially era. 

designed rotating chopping filter is used. This chopping filter As a general proposition, the chopping filter 28 provides 
takes advantage of the unique property of the pyroelectric 5 the same function as the prior art chopper, but adds wave- 
detector and will be described hereinafter in more detail. length specificity or spectral filtering. Hie chopping filter 28 

In accordance with the present invention, the chopping includes two distinct filter areas which provide alternate 
filter, in conjunction with the pyroelectric infrared video scans at a measuring wavelength and at a reference wave- 
camera and a special image processor, provides an image of length. All other considerations of chopping are still present, 
the gas doud.TWs image is conihinedw^ 10 though, including the need for flicker processing which 

light video image to provide an actual, real-time picture of comes from the alternating field inversions. The chopping 
a gas cloud. The gas cloud can actually be seen drifting filter 28 is constructed specifically for detecting a particular 
through a plant or other facility. This video image can be gas. The measuring wavelength is selected as a strongly 
captured with a video cassette recorder to be reviewed and absorbing infrared wavelength for the gas in question. The 
analyzed at a later date. 15 reference wavelength is selected as a weakly absorbing 

The principles by which a gas can be detected using a infrared wavelength for the gas in question, preferably at a 
filtered infrared video camera in accordance with the present wavelength close to the measuring wavelength. The filter 28 
invention is shown in FIG. 1. A gas cloud 10 is viewed is men constructed with alternating band pass filter areas 
against a background object 12, which can be the sky, toned to the measuring and reference wavelengths. Trans- 
buildings and the like. As long as there is an effective 20 missions of radiation at the two different wavelengths pro- 
temperature differential between the background tempera- duce the differential signal needed for the operation of the 
ture T B and the gas cloud temperature T G , there will be either pyroelectric camera. The preferred closeness of the measur- 
absorption or emission of radiation by the gas cloud and the ing and reference wavelengths will minimize imaging from 
gas cloud will have unique spectral properties. The gas cloud objects having large temperature differentials from the 
produces a spectral unbalance since it absorbs radiation at 25 environment, but not related to the gas in question. The two 
certain characteristic wavelengths but not at other wave- different filter areas on the chopping filter 28 are represented 
lengths and replaces it with radiation related to the tempera- in FIG. 2 as areas A and B. 

tore of the gas. If a camera 14 is fitted with a filter 16 that A preferred construction of the chopping filter is shown in 
matches a strongly absorbing wavelength of a particular gas, HG. 3. The filter is a thin, circular wafer or substrate of 
a sharp change in energy will be noted if mat gas is present 30 silicon or other semiconductor material and includes two 
Only in the extreme case where effective T B =effective T G separate filter sections in a type of yin-yang shape, also 
will no differential signal exist, preventing the gas from referred to as a pair of nested, outwardly tapering archime- 
being imaged. In practice, particularly in industrial dian spirals separated by about 180°. The filters are con- 
applications, significant temperature differentials nearly structed by known vapor deposition techniques in which 
always exist. 35 multiple layers of certain mate rial g at particular thicknesses 

In a complete system, the gas imaging infrared camera is are deposited on the substrate to form band pass filters at the 
combined with a visible light video camera, both preferably desired wavelengths. One filter area is designed to pass 
included in camera 14, and both video signals are supplied radiation at the measuring wavelength and the other filter 
to an image processor 18. The radiation transmitted to the area is designed to pass radiation at the reference wave- 
visible light video camera is not subjected to the special. 40 length. The twin spiral shape is advantageous in facilitating 
filtering represented by filter 16. As will be explained a desired top to bottom wiping action across the thermal 
hereinafter in more detail, the output signal of the infrared imag ing material of the pyroelectric infrared camera. (The 
video camera is processed to reduce noise and enhance the position of the detector with respect to the filter is shown in 
image detected thereby, and is then combined with the signal FIG. 3 as a circle identified by reference number 36.) This 
generated by the conventional, visible light video camera. 43 wiping action duplicates that of a standard chopper for the 
Hie processor 18 supplies its output signal to a monitor 20 same reason of inducing change information into the tube, 
or the like, which displays the gas cloud as a brightly colored This shape of the chopping filter 28 is also preferred since 
overlay 22 superimposed on a black and white image 24 the electron scanning beam in the pyroelectric vidicon 
from the visible light video camera. This final presentation camera version must first track one wavelength and then the 
of elements identifies the position; size, direction and mag- so other. The rotational frequency of the chopping filter 28 is 
nitude of the leak or gas cloud directly against the back- determined by the scanning rate of the pyroelectric camera, 
ground from which it emanates, This exact positional iden- For each field scan, the electron beam must read off signals 
tification allows the source of the leak to be pinpointed as which are due to one wavelength. The shape of the filter 28 
well as show the extent of the diffusion of the gas cloud. ensures that these conditions are satisfied. 

The principles of operation of the present system are 55 When the pyroelectric camera is fitted with the chopping 
shown in more detail in FIG. 2. The infrared energy radiated filter 28 discussed above, the camera functions as a spec- 
by the gas cloud 10 passes through a lens 26 and through a trophotometer in that it responds only to changes at particu- 
rotating chopping filter 28 to a pyroelectric detector 30 lar wavelengths. This wavelength sensitivity is the heart of 
(vidicon tube or detector array) in the pyroelectric camera. the ability of the present invention to detect chosen gases. 
The filter 28 is rotated in a controlled manner by a motor 32 60 The pyroelectric camera detects the presence of particular 
contained within the housing for the pyroelectric camera. An gases by measuring the differential absorption or emission in 
advantage of this arrangement is that an on-off chopper filter the infrared region. First one filtered image is exposed to the 
provided in standard pyroelectric infrared video cameras can image tube and then the other. This process is repeated at 
be readily removed and replaced by the chopping filter 28 of rates rapid enough to convey the gas image as a real-time 
the present invention. The signals developed by the pyro- 65 video picture which can be seen on an ordinary video 
electric detector 30 are subjected to standard monitor. If the two filters are properly matched and the 
synchronization, amplification and difference signal background contains no emissivity differences or thermal 
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a. 4 ,dJfie^j^ v &en ,^qduiigas^c^ L T^. is,because there ; is,nQ; , ^temperatures* the temperature, dependence's such that effect 

change at all between the two filtered scenes and thus, no tive equality in radiance can be achieved over a moderate 

different signal. By knowing which particular wavelengths range of temperatures. 

arc characteristic for each gas it is possible to readily design The video image processor 18 shown in FIG. 1 is used to 

the two filter areas such that one is at a peak absorption 5 enhance the output image of the infrared video camera and 

wavelength of the gas while the other is at an unaffected to <l udl me inevitable noise developed in the system. All 

wavelength. sensitive detectors and amplifiers, such as the system of the 

_ " . -.^ _„ . . . „ ^ . present invention, show significant noise when operated at 

u^° P r^ , should be made to exacting oler- ^ ^ ^ s needed for sudl sensMve tio|L ^ e ^ 

ances both physicaUy and opUcaUy. The mechanical balance is to characterize the differences between the signal and the 

must be as close to perfect as possible in order to minimize io noisc and cxtract ^ particularly when the 

vibration. The optical densities of the filter areas should be unwanted noise can exceed the desired signal by magnitudes 

set so that the energy balances out when there is no gas 0 f 1,000 or more. This extraction process is accomplished by 

present, producing a net output of zero from the detector. identifying what is consistent about the noise and what is 

This is important because the wavelengths of the two filter different about the signal. The unique and known qualities of 

areas lie at different points on the black body curve, as 15 the signal are used as identifiers to sift the noise from the 

shown in FIG. 4, and cannot be equal in energy to each other. signal. The signal from the infrared video camera, when 

The density of one section should be carefully adjusted imaging in the presence of gas, has several such identifiers 

relative to the other section so that the effect of the black that may be exploited to remove noise from the camera's 

body curve is minimized. output signal The computerized processing in the image 

The operation and requirements of the chopping filter 28 20 processor makes it possible to produce a crisp, clean image 

can also be described mathematically. The spectral radiance even k verY adverse signal-to-noise ratio conditions, 

of the scene, which includes both gas at a temperature T c The first identifier used in the present system is the gas 

and a background at a temperature T B is given by the cloud s property of constat change or constant niovement; 

following equation: * te boundaries are constantly in a state of flux. This constant 

23 change can be detected by subtracting image frames from 
each other. When frames are subtracted from each other, 

N(K T)=N < Ai-e-° cl y+Nae** CL (i) nothing that is constant remains; only changes are carried 

forward. This operation is very useful in image processing 

Where Nis the spectral radiance of the scene, G andB refer bccausc it climmatcs feed pattcrn noise or differential 

to the gas and background, respectively, a is the absorption 30 siglial ^ t0 background objects in the picture. Since the 

coenlcient C is the gas concentration and Lis the gas cloud objective of the infrared video camera is to image gas clouds 

length. Atmospheric absorption has been ignored in equation only ^ background objects are considered to be noise. After 

(1) above. The first term in equation (1) is spectral radiance ^ processing step, only gas cloud changes and random 

of the gas re-emitting the amount absorbed from the back- noise ^ reinain ^ ^ picture. This is a great reduction of 

ground. The second term in equation (1) is the spectral 35 inf orma tion to be processed in further steps, 

radiance of the background after absorption by the gas. Another special characteristic of flowing gas is that it has 

As discussed above, the filter areas on the chopping filter a &oupcd OT ^obby nature", or an appearance, once it is 

28 have their transmissions peaked at a measuring wave- made visiD ] c , that can be referred to as "cloud-like" In 

length which is absorbed by the gas and at a reference essence, this means that while the boundaries of the gas 

wavelength which is not absorbed by the gas. Since the 40 doud m constant i y changing, its interior is relatively con- 

mfrared camera responds to difference in spectral radiances, sistent ^ it ^ continuous into In^cr area and may be 

the relevant equation is: detected because of that feature. Random noise, on the other 

hand, has a nature of non-continuous behavior and changes 

AN^^w^WN^yNcfr^e-***- (2) randomly from one moment to the next. Several processing 

45 steps take advantage of this cloud-like nature of the gas. 

where X M is the measuring wavelength and X R is the The preferred image processor used in this system is 

reference wavelength far the particular chopping filter. shown in detail in FIG. 5. As dliscussed above, a 

The contrast is the difference between the target radiance conventional, visual light video camera, such as a black and 

and the background radiance divided by the sum of the white video camera 40, is positioned preferably co-located 

radiances. For the chopping filter of the present invention, 50 close to or adjacent an infrared video camera 42. Other 

the contrast R is given by: physical arrangements for the video cameras 40, 42 are 

possible, such as separating the cameras, or using a shared 

[l - <r aCL \ (3) lens and beamsplitting arrangement with combined cameras, 

~ NcOw) (l - + NbQw) (i + er* 0 ) - 2Nb(Kr) as long as the two video cameras 40 and 42 view substan- 

•nn. *u «r * *. . ^_ 55 tially the same scene. The output signal developed by the 

When the effect temperas dtference be^reen the gas ^ ^ wWte ^ ^ 'LierHo, representing the 

m / £ ^ L * ' W ^ Nc<M=Nfl( ^ ^ visible image of the scenes supplied directly to a graphic 
^H^U then R=0. For high gas concentrates. ove ^^ whlch ^ of ^ck 

whene =0. then the contrast is given by: and wWte camera 40 with the outpw sigi^ of fte inftaxed 

tfcOt^ - Nfiku) (4) 60 camera 42 after * e infrared camera output signal has been 

r = j^^u^^-^V J U^ subjected to processing as will be discussed hereinafter. The 

combined output signals from the black and white camera 40 

This contrast is maximized by having N^A^H^*). This and the infrared camera 42 are then supplied to a color video 

effect can be achieved by adjusting the transmission of the monitor 46 which provides the desired view of the gas cloud 

filter areas to allow for the difference in the Planck function 65 surjerimposed on the background view observed by the 

at the two wavelengths, referring once again to FIG. 4. cameras. In a preferred embodiment, the gas cloud will be 

While this can be achieved precisely only for specific shown as a red or brightly colored image superimposed upon 
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provides a stark and readily apparent image on the monitor block 56 functions to subtract the currently detected signal, 

46 for easily detecting the location, size and movement of as supplied from the integrator 52, from a stored past 

the gas cloud. reference signal in the dual frame store 54 on a continuous 

Since the processor is preferably contained on printed 5 frame-by-frame basis, 
circuit boards which operate in conjunction with a high The output of the difference block 56 is supplied to a 

speed digital computer, the output signal generated by the threshold detector 58 which compares the output of the 

infrared camera 42 is first digitized through an analog-to- difference block 56 to a preset threshold level. The threshold 

digital converter 48. The output of this converter 48 is a detector 58 determines what magnitude of difference detec- 

digitized 256x256 pixel array. 10 tion is significant and should be considered as a possible 

A consequence of running a sensitive gas camera at very valid gas cloud signal for further processing, 
high gains and comparing subsequent pictures over time is The signal that represents the gas cloud, which the image 
that very minor drifts can cause significant and erroneous processor is attempting to isolate, is represented by certain 
results. These drifts are almost impossible to avoid over the consistencies that noise accompanying the signal does not 
temperature ranges and periods of time to which the infrared 15 have. These consistencies exist both spatially and tempo- 
camera 42 is subjected. To overcome this problem, the rally. By constructing a processor tuned to these 
present system includes a form of automatic compensation consistencies, the desired signal can be almost totally iso- 
that keeps the camera stabilized over time. As shown in FIG. lated from the noise with little loss of signal In order to take 
5, the output of the analog-to-digital converter 48 is fed back advantage of these consistencies of a gas cloud, the output 
through a whole picture average block 50 to the infrared 20 of the threshold detector 58 is supplied to a convolution 
camera 42. The whole picture average block 50 functions to block 60 which conducts a spatial filtering process on the 
calculate a whole picture average continuously so that a base signal. The output of the convolution block 60 is then 
line can be measured. The stability of the camera can be supplied to a temporal contiguous block 62 for temporal 
maintained in real-time by using the black level control processing of the signal. 

input on the infrared camera 42. Basically, the whole picture 25 The spatial filtering process carried out in convolution 
average block 50 functions as a way to monitor the overall block 60 is based upon the principle that a gas is usually' 
picture brightness level. It works by summing the amplitude gathered together into a group or a block, which allows 
values of all of the pixels in each video field and dividing the filtering to be accomplished on areas of high spatial con- 
sum by the number of fields used to make this sum. The centration. To accomplish this in the present invention, 3x3 
whole picture average block 50 is useful in the present 30 pixel areas are swept across all areas of the image in each 
invention in that the presence of gas will modify the sum to frame in order to detect grouped areas and certain spatial 
a higher or lower value than it was before the introduction shapes of groups* There are various methods which could be 
of the gas. II is also useful in regulating the long-term drift used for detecting the desired spatial characteristics. One 
of the infrared camera 42. method would be to count the number of activated or fired 
The whole picture average block 50 can be defined as a 35 pixels in mis nine pixel area. A high count, such as having 
computerized control of the black level of the infrared five or six of the nine pixels fired, would be indicative of a 
camera 42 in a servo loop which regulates the infrared blob or a gas cloud. In another technique, specific shapes 
camera 42 closely over time. Any drift in black level is would be sought within the 3x3 pixel area rather than just 
sensed immediately by integration circuitry in the whole relying upon the fired pixel count For example, three pixels 
picture average block 50 and is corrected by means of a 40 in the block representing a vertical column can be sought 
digital-to-analog converter therein which has its output fed This technique allows for a low count with greater sensi- 
back into the black level control circuits of the infrared tivity while being insensitive to noise which is predomi- 
camera 42. The chief significance of controlling the black nantly a horizontal phenomenon in the raster scanning which 
level of the infrared camera 42 is the ability to subtract accompanies the generation of the frames. When noise 
camera frames from a single image taken at an earlier time. 45 occurs, it is likely to involve several pixels on a line adjacent 
This operation permits extreme sensitivity to even the slow- to each other because of ringing and other transmission line 
est moving of features that are totally lost by subtracting type effects. This type of noise artifact would never manifest 
consecutive frames from each other. If the natural drift of the itself in adjacent raster lines because of the long time 
infrared camera 42 were to remain unchecked, significant interval between the lines. 

information regarding a gas cloud might be overlooked since 50 The preferred method for carrying out the spatial filtering 

the difference between past images and the present image in the convolution block 60 is shown in FIGS. 6A and 6B. 

would always be significant just from the camera drift itself. In FIGS. 6A and 6B, a small open circle represents an 

The output signal of the analog-to-digital converter 48 is unfiled pixel in the pixel array, while a solid circle represents 

then supplied to an integrator 52 which integrates two or an active or fired pixel representing a portion of the image 

more successive frames together before further processing is 55 under detection which satisfies the earlier steps of the 

carried out Noise can be significantly diminished without processor. A solid square has been drawn around a 3x3 array 

harming the desired signal by integrating the signal over of pixels. For determining whether to fire one of the pixels 

several frame intervals. Since noise is a random event, in this block, such as the pixel in the upper left-hand corner, 

integration reduces it The output of the integrator 52 is the convolution block 60 will examine the left-hand-most 

supplied to a dual frame store 54 which holds a selected, past 60 vertical column, identified by a surrounding dashed rect- 

reference frame to which subsequent input frames are com- angle. Under the criteria established in the present invention, 

pared. a significant or gas related event is considered to have 

As discussed above, one basic concept of the present occurred only if the three vertical pixels in the left-hand- 
invention is that frame subtraction will give the needed most column of the 3x3 pixel array are all active or fired. As 
movement data for the gas cloud. As shown in FIG. 5, the 65 shown in FIG. 6A, only one of the pixels within the 
output of the integrator 52 is supplied also directly to a left-hand-most vertical column is fired. This will result in the 
difference block 56, and the output of the dual frame stare upper left-hand-most pixel of the block remaining not fired 
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,0*;. activated after; the spatial filter processing m me -conyo- , ■■>, .becomes recognized a,s significant, and should be.setas.low , » 

lution block 60. While this example does look at the upper as possible without saturating the picture with noise. The 

left-hand-most pixel of die 3x3 array, a determination far pixel amplitude resolution of the system is 256 bits which, 

firing any of the pixels in this array can be used by the when subtracted from similar resolution pixels, yields a 

left-hand-most vertical column. It is only important that the 5 difference value. By setting the value of the threshold 

same pixel within the 3x3 array be used for all of the control 74 to a low number, nearly any difference between 

subsequent evaluations. In the 3x3 array of pixels shown in corresponding pixels can be detected. Likewise, the integral 

FIG. 6A, the left-hand-most column has all three of its pixels control 70 can be set during operation to diminish the noise 

active or fired This would cause the upper left-hand-most in the image. 

pixel of the 3x3 array to be fired. 10 A kernel control 76 on the control panel 68 is used to 
The temporal processing carried out in the temporal select which of various predetermined spatial filtering pro- 
contiguous block 62 is similar to the spatial processing cesses will be conducted on the signal. Some of the various 
except that it responds to the grouping of pixels over time options were outlined above, with the preferred spatial 
rather than in space. The underlying theory is that gas cloud filtering being the presence of three fired pixels in the 
related pixels should exist over a sustained period of time if 15 left-hand-most vertical column of a 3x3 pixel array. This 
they are true gas signals, and would not if they were created 3x3 array was selected because it accomplishes good results 
from noise. Processing in the temporal contiguous block 62 in detecting gas clouds, but does not require undue computer 
fires a particular pixel in a particular frame only if the prior processing speed and memory. Larger pixel arrays, such as 
processing steps determined mat an event occurred in that 4x4, 5x5 and the like can be used, and also other patterns 
pixel for a certain number of previous frames. Only if the 20 and/or combinations of fired pixels can be used. The various 
event occurred in the desired number of frames would the selected options would be programmed into the computer 
particular pixel be fired for further processing. and selected by the kernel control 76. 

The output from the temporal contiguous block 62 is then Similarly, the number of frames that are used in the 

supplied to a sustain er 64 which maintains each pixel as fired temporal contiguous block 62 to indicate a significant event 

or active for a particular period of time. It was discovered 25 can be set by the scan control 78. A response can be made 

that a fired signal, once detected, occasionally did not to fired pixel sequences as short as one or as long as sixteen 

provide a continuous, smooth display. Instead, the display or more pixels in a row. A temporal contiguous test carried 

showed a spotty or blotchy output This visual defect is out over about eight subsequent frames has been found to be 

overcame with the addition of the sustainer 64 which satisfactory for detecting gas clouds. Although decisions can 

effectively extends the display for several subsequent video 30 be made based upon responses that occur over durations 

frame intervals. This feature is used solely for the improve- longer than sixteen pixels in a row, this method might be 

ment of the appearance of the display and does not affect the useful only in situations of extreme noise where holes or 

extraction of the signal which occurred in the earlier pro- non-fired pixels might be made in an otherwise valid signal 

cessing steps. The output of the temporal contiguous block Finally, a decay control 80 is used to set the duration for the 

62 is also supplied to an event counter 66 which keeps track 35 sustainer 64. 

of the occurrence of "events" or fired pixels which satisfy all The present system can be readily incorporated into a 

of the processing requirements. complete system as shown in FIG. 7. The video signals 

The output of the sustainer 64, which is basically the end developed by the black and white camera 40 and the infrared 

result of the processing of the output signal from the infrared camera 42 are supplied directly to a computerized video 

camera 42, is supplied directly to the graphic overlay 44. As 40 processor 90 as described above in connection with FIG. 5. 

discussed above, the graphic overlay 44 combines the pro- Appropriate DC or AC power is supplied to the video 

cessed output from the infrared camera 42 with the output processor 90. In addition, a field operator control panel 92 

signal of the black and white camera 40 and supplies this can be used to set the adjustable parameters for the video 

combined signal to the monitor 46. processor 90, as well as supply other data and information 

Each of the integrator 52, dual frame store 54, threshold 45 thereto. The output signals developed by the computerized 

detector 58, convolution block 60, temporal contiguous video processor 90 can be supplied directly to a display 

block 62 and sustainer 64 have controls associated therewith monitor 94 which can also have a video cassette recorder 

on a remote or adjacent control panel 68 or the like. An attached thereto or associated therewith for recording the 

integral control 70 supplies a signal to the integrator 52 detected signals for further analysis. In addition, the output 

which sets the number of frames which will be integrated 50 signals developed by the video processor 90 can be supplied 

before further processing takes place. The integral control 70 directly to a remote control computer 96 for data storage and 

can be set from one frame up to nine frames or more, later analysis, particularly where unattended monitoring is 

although it has been found that fewer number of frames need used. The system can also be designed to include automatic 

be integrated for adequate processing, The rate control 72 is alarms and the like for clearly indicating when certain 

used to set how often a reference frame is stored in the dual 55 conditions are detected. 

frame store 54. This rate can be set to as frequently as once The system discussed above is a passive camera system 

per frame, although such a rapid setting will remove the which is capable of imaging toxic gas or other gas releases 

longer term effects of drift and the like. If the rate is set at in normal working environments. Presently, the system has 

a slower rate, for updating at longer intervals, the system achieved an imaging capability against temperature differ- 

becomes extremely sensitive because long-term changes are 60 entials of around 5° C. with a sensitivity in the range of fifty 

not eliminated and even small amounts of slow building gas ppra-meters. Sensitivity of one ppm-meter has been 

become detectable. achieved against heated or cooled background objects. 

Once processing is under way. the operator can adjust the While the system discussed above is a preferred arrange- 

threshold level by a threshold control 74, which can be a ment for imaging invisible gases, the video image processor 

potentiometer supplying a signal level to the threshold block 65 discussed can be also used for imaging clouds of visible gas. 

58. The threshold level set by the threshold control 74 such as steam, smoke and the like by using a black and white 

reflects the value at which the difference image data camera in conjunction with the processor without an infrared 
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^video.; camera^ In -addition, the image processor^ ■• can ^be^ 'representing , an image of a ■ gas* xIoikI-* superimposed- - -^^^-.^h ; 

combined with a cooled infrared video camera, which has a upon an image of said background scene; and 

fixed filter across its input aperture. All of the advantages of f . display means for displaying the signal generated by 

the processing discussed above would be available in that said overlay means. 

arrangement Although it is preferred overall to use an 5 %. The apparatus of claim 1 wherein said infrared video 

uncooled, pyroelectric infrared video camera in this system, camera means is a pyroelectric infrared video camera, 

with the specially designed chopping filter, the advantages 3. The apparatus of claim 1 wherein said second bandpass 

of the processor of this invention would also be obtained by filter passes infrared radiation at a wavelength close to the 

using it with other infrared video cameras. Moreover, the wavelength of said first band pass filter, 

chopping filter can be used with Don-pyroclectric infrared 10 4. The apparatus of claim 3 wherein said chopping filter 

cameras, such as photon detector arrays, but the system is a rotating disc having separate first and second band pass 

would need to subtract the signals generated by the different areas therein. 

filter areas to develop a difference signal. This difference 5. The apparatus of claim 4 wherein said chopping filter 

signal generation or waveband comparison is carried out i s a thin, circular wafer of a semiconductor substrate having 

automatically in pyroelectric infrared video cameras, 15 layers of material deposited thereon to form said first and 

Furthermore, the present invention cotfemplates the use of second band pass filter areas at said strongly and weakly 

the combined pyroelectric infrared video camera and chop- absorbing infrared wavelengths, respectively, 

ping filter, but with any other video processors. & The apparatus of claim 5 wherein said band pass filter 

Having described above the presently preferred embodi- areas of said chopping filter are formed in the shape of 

ments of this invention, it is to be understood mat it may be 20 nested, outwardly tapering archimedean spirals separated by 

otherwise embodied within the scope of the appended about 180°. 

claims. 7. The apparatus of claim 1 wherein the chopping filter 

We claim: satisfies the equation: 

1. Apparatus for generating a visible image of a gas cloud 

against a background scene, said apparatus comprising: 25 

a. visible light video camera means for acquiring a visible 

light video image of a scene; where, N ff is the spectral radiance of the background scene, 

b. infrared video camera means for acquiring an infrared ^ the wavelength characteristic of the first band pass 

video image of substantially the same scene as said ^ ^ d ** fe ±c wavelength characteristic of the second 

visible light video camera means; 30 band pass mten 

. . £lA . , . . . 8. The apparatus of claim 1 further including a threshold 

c. chopping filter means assoaated with sari infrared ^ to &e mmacc signal from ^ ^ 

video mm 1 means far alternate^ presenting band paris0I1 ^ for mpp lyin& to said convolution means only 

pass filters w^ch pass rad^on of two different Infra- ^ ddifference signa K<£ exceed a r^detennined thresh- 

red wavelengths to said infrared video camera means, 35 Q ^ level ^ 

^ fif 1 f ^ band pass filters passing radiation at 9>The apparatus ^ claim 8 further including an integrator 

a strongly absorfnngjnfrared wavelength far a prede- mea ns response to me output of said iin^ vioeo camera 

termined gasand with a second of said bandpass filtos means for 1 ? tegratillg at ^ of said 

passing infrared radiation at a weaHy absoitang infra- ^ eam ^^ ou ^ at be fore su lying a ^ for 

red wavelengthfor said predetermined gas; ^ storage in said storage means and before supplying a current 

d. image processing means responsive to an output from frame to said comparison means. 

said infrared video camera means for reducing noise 10. The apparatus of claim 1 further including a sustainer 

inherent therein and enhancing an information signal means responsive to an output from said temporal means for 

therein generated by a detected gas cloud, with said maintaining each activated pixel in a frame as activated for 

image processing means including: 45 a predetennined period of time beyond the duration of said 

i. storage means for storing at least one prior output frame. 

frame of said infrared video camera means; u. The apparatus of claim 10 further including control 

ii comparison means for comparing a current output means for establishing the rate of frame integration for said 

frame of said infrared video camera means with a integrator means, the rate of storage of frames in said storage 

prior frame in said storage means and generating a 50 means, the threshold level of said threshold means, the 

difference signal representing differences between spatial filtering method used in said convolution means, the 

said current frame and said prior frame; number of consecutive prior frames considered in said 

iii. convolution means responsive to said difference temporal means, and the period of time of said sustainer 
signal from said comparison means for determining, means. 

on a pixel by pixel basis for each frame, whether a 55 \2, The apparatus of claim 1 wherein said convolution 

particular pixel should be further activated based means examines a spatial relationship of an nxn array of 

upon a spatial relationship of activated pixels in a pixels, where n is an integer greater man 1, including the 

surrounding matrix of pixels, including said particu- particular pixel under consideration, 

lar pixel; and 13^ The apparatus of claim 1 wherein said convolution 

iv. temporal means responsive to an output generated ^ me ans examines a 3x3 array of pixels, including the par- 
by said convolution means for activating a pixel only ticular pixel under consideration. 

if that pixel is activated in the current frame and was 14, The apparatus of claim 13 wherein said particular 

activated in a predetermined number of consecutive ^ a frame i s activated only if the three pixels in a 

prior frames; left-hand column of said pixel array are each activated. 

e. overlay means for combining an output from said 65 15. The apparatus of claim 1 further including, in a 
visible light video camera means with an output from feedback loop from said infrared video camera means output 
said image processing means and producing a signal to said infrared video camera means, a whole picture aver- 
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,rwu .^.r vager,4 4 which : monitors ; a baseline brightness , ieyeL,of; isaicb * 
infrared video camera means and, in response to any drift in 
said baseline brightness level adjusts a black level control 
in said infrared video camera means to maintain an overall 
brightness level in said infrared video camera means at a 5 
substantially constant level 

16. The apparatus of claim 15 wherein said baseline 
brightness level is generated by summing the amplitude 
values of all pixels in one or more frames and dividing the 
sum by the number of frames utilized. 1Q 

17. Apparatus for processing a multiple pixel output 
generated by a video camera to reduce unwanted noise and 
enhance an underlying information signal contained in said 
output, said apparatus comprising: 

a. storage means for storing at least one prior output frame 
of said video camera; 

b. comparison means for comparing a current output 
frame of said video camera with a prior frame in said 
storage means and generating a difference signal rep- 
resenting differences between said current frame and 
said prior frame; 20 

c. convolution means responsive to said difference signal 
from said comparison means for deterrriining, on a 
pixel by pixel basis for each frame, whether a particular 
pixel should be further activated based upon a spatial 
relationship of activated pixels in a surrounding matrix 25 
of pixels, including said particular pixel; and 

d. temporal means responsive to an output generated by 
said convolution means for activating a pixel only if 
that pixel is activated in the current frame and was 
activated in a predetermined number of consecutive 30 
prior frames. 

18. The apparatus of claim 17 further including a thresh- 
old means responsive to a difference signal from said 
comparison means for supplying to said convolution means 
only said difference signals which exceed a predetermined 35 
threshold level. 

19. The apparatus of claim IS further including an inte- 
grator means responsive to the output of said video camera 
for integrating at least two frames of said video camera 
output before supplying a frame for storage in said storage 40 
means and before supplying a current frame to said com- 
parison means. 

20. The apparatus of claim 17 further including a sustain er 
means responsive to an output from said temporal means for 
maintaining each activated pixel in a frame as activated for 45 
a predetermined period of time beyond the duration of said 
frame. 

21. The apparatus of claim 20 further including control 
means for establishing the rate of frame integration for said 
integrator means, the rate of storage of frames in said storage 50 
means, the threshold level of said threshold means, the 
spatial filtering method used in said convolution means, the 
number of consecutive prior frames considered in said 
temporal means, and the period of time of said sustainer 

... means. 55 

22. The apparatus of claim 17 wherein said convolution 
means examines a spatial relationship of an n xn array of 
pixels, where n is an integer greater than 1, including the 
particular pixel under consideration. 

23. The apparatus of claim 17 wherein said convolution 60 
means examines a 3x3 array of pixels, including the par- 
ticular pixel under consideration. 

24. The apparatus of claim 23 wherein said particular 
pixel in a frame is activated only if the three pixels in a 
left-hand column of said pixel array are each activated. 65 

25. The apparatus of claim 17 further including, in a 
feedback loop from said video camera output to said video 



■ camera, a, whole picture averager whichjnomtors a baseline,, 
brightness level of said video camera and. in response to any 
drift in said baseline brightness level, adjusts a black level 
control in said video camera to maintain an overall bright- 
ness level in said video camera at a substantially constant 
level. 

26. The apparatus of claim 25 wherein said baseline 
brightness level is generated by summing the amplitude 
values of all pixels in one or more frames and dividing the 
sum by the number of frames utilized. 

27. Apparatus for generating a visible image of a gas 
cloud comprising: 

a. a video camera; and 

b. image processing means for processing a multiple pixel 
output generated by said video camera to reduce 
unwanted noise and enhance an underlying information 
signal representing said gas cloud, said image process- 
ing means including: 

i. storage means for storing at least one prior output 
frame of said video camera; 

ii comparison means for comparing a current output 
frame of said video camera with a prior frame in said 
storage means and generating a difference signal 
representing differences between said current frame 
and said prior frame; 

iiL convolution means responsive to said difference 
signal from said comparison means for determining, 
on a pixel by pixel basis for each frame, whether a 
particular pixel should be further activated based 
upon a spatial relationship of activated pixels in a 
surrounding matrix of pixels, including said particu- 
lar pixel; and 

iv. temporal means responsive to an output generated 
by said convolution means for activating a pixel only 
if that pixel is activated in the current frame and was 
activated in a predetermined number of consecutive 
prior frames. 

28. The apparatus of claim 27 wherein said video camera 
is an infrared video camera. 

29. The apparatus of claim 27 wherein said video camera 
is a pyroeiectric infrared video camera. 

30. The apparatus of claim 27 wherein said video camera 
is a visible light video camera. 

31. The apparatus of claim 27 further including a thresh- 
old means responsive to a difference signal from said 
comparison means for supplying to said convolution means 
only said difference signals which exceed a predetermined 
threshold level 

32. The apparatus of claim 31 further including an inte- 
grator means responsive to the output of said video camera 
for integrating at least two frames of said video camera 
output before supplying a frame for storage in said storage 
means and before supplying a current frame to said com- 
parison means. 

33. The apparatus of claim 27 further including a sustainer 
means responsive to an output from said temporal means for 
maintaining each activated pixel in a frame as activated for 
a predetermined period of time beyond the duration of said 
frame. 

34. The apparatus of claim 33 further including control 
means for establishing the rate of frame integration for said 
integrator means, the rate of storage of frames in said storage 
means, the threshold level of said threshold means, the 
spatial filtering method used in said convolution means, the 
number of consecutive prior frames considered in said 
temporal means, and the period of time of said sustainer 
means. 
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> ^35/ The apparaws of *claim 27 wherein-said -^convolution' 
means examines a spatial relationship of an nxo array of 
pixels, where n is an integer greater man 1, including the 
particular pixel under consideration. 

36. The apparatus of claim 27 wherein said convolution 
means examines a 3x3 array of pixels, including the par- 
ticular pixel under consideration. 

37. The apparatus of claim 36 wherein said particular 
pixel in a frame is activated only if the three pixels in a 
left-hand column of said pixel array are each activated. 

38. The apparatus of claim 27 further including, in a 
feedback loop from said video camera output to said video 
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camera^ a' whole pi 

brightness level of said video camera and, in response to any 
drift in said baseline brightness level,, adjusts a black level 
control in said video camera to maintain an overall bright- 
ness level in said video camera at a substantially constant 
level. 

39.. The apparatus of claim 38 wherein said baseline 
brightness level is generated by summing the amplitude 
values of all pixels in one or more frames and dividing the 
sum by the number of frames utilized. 
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